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DETAILED ACTION 

1 . This Office Action is in response to an amendment filed 06/26/2006 with a priority 
date of 10/03/2003. 

2. Claims 1-48, and 50-65 are currently pending. Claims 1, 22, and 48 are 
independent claims. 

Claim Rejections • 35 USC § 102 

3. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

4. Claims 1, 2, 4, 48, and 50 remain rejected under 35 U.S.C. 102(b) as being 
anticipated by Yamashita et al. (hereinafter Yamashita, U.S. Patent No. 5,555,362 filed 
07/26/1995, issued 09/10/1996). 

In regard to independent Claim 1 (and similarly independent Claim 48), 
Yamashita teaches receiving a definition of at least one region in an image in that a 
document image is scanned by the image input unit (2), and character strings, vertical 
and horizontal black lines, and other black pixel regions (picture-element) are extracted 
from the image and stored in the image memory 3 (Step 21) (Col. 3, lines 35-40). 

Yamashita also teaches the region definition having a location specification and a 
type specification in that subsequent processing is executed in accordance with 
extracted rectangle data. Then, area segmentation of the document image is 
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automatically executed by the automatic area segmentation unit 4A of the area 
generation unit 4 (Step.22). First, long, wide, and White pixel regions (type specification) 
and long black lines (type specification) to serve as separators for objects are extracted 
from the x.y-coordinates of the rectangle (position). Then, graphic areas (type 
specification) are removed before character areas (type specification) are roughly 
segmented using the extracted separator (Col. 3, lines 41-49). 

Yamashita also teaches displaying the boundaries of the at least one defined 
region according to its type specification (see Fig. 3). 

Yamashita also teaches receiving a definition of a visible area in the image, the 
visible area definition having a specification of margins around the image; and 
generating an image layout definition comprising the region definition and the visible 
area definition (see Fig 6, layout created from segmentation analysis of image positions 
and components). 

In regard to dependent Claim 2, Yamashita teaches displaying the image on a 
display (Col. 3, lines 53-59; Fig. 6). 

In regard to dependent Claim 4 (and similarly independent Claim 50), 

Yamashita teaches automatically determining the definition of the at least one region in 
the image by segmentation analysis of the image (see Figs. 2 and 3, Fig. 2 step 22). 
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Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

6. Claims 12-14, and 58-60 remain rejected under 35 U.S.C. 103(a) as being 
unpatentable over Yamashita. 

In regard to dependent Claim 12-14 (and similarly dependent Claims 58-60), 

Yamashita fails to teach such limitations for manipulating scanned document images for 
purposes of displaying or transmitting in order to provide an image that is the 
appropriate size, dimension, color depth for the given action. However, such functions 
were known and obvious to one of ordinary skill in the art at the time of invention 
particularly with respect to graphical user interfaces where one would have desired to 
view entire images on a screen independent of the size of the actual image for purposes 
such as identifying regions of interest. 
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7. Claims 3,11,17, and 57 remain rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Yamashita in view of Revankar et al. (hereinafter Revankar, U.S. 
Patent No. 5,767,978 filed 01/21/1997, issued 06/16/1998). 

In regard to dependent Claim 3, Yamashita fails to teach receiving a definition 
of at least one region in an image further comprises receiving a modality specification. 
However, Revankar teaches image segmentation according to classes of regions that 
may be rendered according to the same imaging techniques. Image regions may be 
rendered according to a three-class system (such as traditional text, graphic and picture 
systems), or according to more than three image classes. In addition, only two image 
classes may be required to render high quality draft or final output images. The image 
characteristics that may be rendered differently from class to class may include half 
toning, colorization and other image attributes (see Abstract). It would have been 
obvious to one of ordinary skill in the art at the time of invention to combine the 
teachings of Yamashita and Revankar as both inventions relate to image segmentation. 
Adding the teaching of Revankar provides the benefit of recognizing region types by 
class and by modality (color, bit depth, etc.). 

In regard to dependent Claim 11 (and similarly dependent Claim 57), 
Yamashita fails to teach receiving a user input indicative of a first vertex and a location 
of a second vertex opposite the first vertex of the visible area on the image. However, 
Revankar teaches image segmentation according to classes of regions that may be 
rendered according to the same imaging techniques. Image regions may be rendered 
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according to a three-class system (such as traditional text, graphic and picture 
systems), or according to more than three image classes. In addition, only two image 
classes may be required to render high quality draft or final output images. The image 
characteristics that may be rendered differently from class to class may include half 
toning, colorization and other image attributes (see Abstract). It would have been 
obvious to one of ordinary skill in the art at the time of invention to combine the 
teachings of Yamashita and Revankar as both inventions relate to image segmentation. 
Adding the teaching of Revankar provides the benefit of recognizing region types by 
class and by modality (color, bit depth, etc.). 

In regard to dependent Claim 17, Yamashita fails to teach receiving user 
specification of region type and region modality. However, Revankar teaches image 
segmentation according to classes of regions that may be rendered according to the 
same imaging techniques. Image regions may be rendered according to a three-class 
system (such as traditional text, graphic and picture systems), or according to more 
than three image classes. In addition, only two image classes may be required to render 
high quality draft or final output images. The image characteristics that may be rendered 
differently from class to class may include half toning, colorization and other image 
attributes (see Abstract). It would have been obvious to one of ordinary skill in the art at 
the time of invention to combine the teachings of Yamashita and Revankar as both 
inventions relate to image segmentation. Adding the teaching of Revankar provides the 
benefit of recognizing region types by class and by modality (color, bit depth, etc.). 
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8. Claims 5, and 51 remain rejected under 35 U.S.C. 103(a) as being unpatentable 
over Yamashita in view of Sakai et al. (hereinafter Sakai, U.S. Patent No. 6,735,740 
filed 03/04/1998, issued 05/11/2004). 

In regard to dependent Claim 5 (and similarly independent Claim 51), 
Yamashita fails to teach automatically determining the definition of the at least one 
region in the image by classification analysis of the image. However, Sakai teaches 
such a limitation (Figs. 10A-C; depict progressive classification of image regions based 
on type). It would have been obvious to one of ordinary skill in the art at the time of 
invention to combine the teachings of Yamashita and Sakai as both inventions relate to 
document image analysis. Adding the teaching of Sakai provides the benefit of 
partitioning an image based on types of content identified in the image. 

9. Claims 6, and 52 remain rejected under 35 U.S.C. 103(a) as being unpatentable 
over Yamashita in view of Ohta (U.S. Patent No. 6,163,623 filed 07/26/1995, issued 
12/19/2000). 

In regard to dependent Claim 6 (and similarly dependent Claim 52), 

Yamashita fails to teach receiving a user input indicative of a point on the image; and 
defining a region encompassing the point using segmentation and classification 
analyses of the image. However, Ohta teaches scanning a document, rendering it to a 
touch display, and allowing the user to manually select a region or regions to further 
process; the drawing of a box is done automatically (Col. 7, lines 46-67; Col. 8, lines 1- 
2). It would have been obvious to one of ordinary skill in the art at the time of invention 



Application/Control Number: 10/679,154 Page 8 

Art Unit: 2176 

to combine the teachings of Yamashita and Ohta as both inventions relate to 
designating regions of documents for further analysis. Adding the teaching of Ohta 
provides the user with a means to easily choose which portions of a document to further 
analyze. 

10. Claims 7, 15, and 53 remain rejected under 35 U.S.C. 103(a) as being 
unpatentable over Yamashita in view of Rangarajan (U.S. Patent No. 5,822,454 filed 
04/10/1995, issued 10/13/1998). 

In regard to dependent Claim 7 (and similarly dependent Claim 53), 
Yamashita fails to teach receiving a user input indicative of boundaries of the region on 
the image; and receiving a user input indicative of region type and region modality 
specifications. However, Rangarajan teaches a conventional set of drawing-like tools 
with which the user can graphically create 311 the user defined zones. This is done by 
choosing an appropriate drawing tool, such as a rectangle or polygon creation tool, and 
applying it to the de-skewed image to select the individual areas or zones containing the 
desired text information. Fig. 7a illustrates one example of a suitable user interface 705, 
showing a de-skewed document 700. Fig. 7b illustrates the same document now 
including a number of user-defined zones 701 . A palette of drawing tools 703 is also 
shown, with various graphical tools for selecting the user-defined zones 701. Once the 
user defines a number of zones, the coordinates of the boundary of each of user 
defined zone is stored, preferably using the coordinates of an upper left hand corner, 
and a lower right hand corner where the user defined zone is a rectangle. For general 
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polygonal user defined zones, the coordinates of each vertex may also be stored (Col. 
9, lines 15-37; Figs. 7A-B). It would have been obvious to one of ordinary skill in the art 
at the time of invention to combine the teachings of Yamashita and Rangaraian as both 
inventions relate to document image analysis. Adding the teaching of Rangaraian 
provides the benefit of manually defining image regions. 

In regard to dependent Claim 15, Yamashita fails to teach receiving definition 
of at least one region comprises receiving a user specification of a location and 
boundaries of a region in the image. However, Rangaraian teaches input of vertices to 
define an image region (Col. 9, lines 15-37; inputting polygons). It would have been 
obvious to one of ordinary skill in the art at the time of invention to combine the 
teachings of Yamashita and Rangaraian as both inventions relate to document image 
analysis. Adding the teaching of Rangaraian provides the benefit of manually defining 
image regions. 
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1 1 . Claims 8-1 0, and 54-56 remain rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Yamashita in view of Rangarajan, and in further view of Revankar. 

In regard to dependent Claim 8 (and similarly dependent Claim 54), 
Yamashita fails to teach receiving a user input indicative of vertices of the region on the 
image. However, Rangarajan teaches input of vertices to define an image region (Col. 
9, lines 15-37). It would have been obvious to one of ordinary skill in the art at the time 
of invention to combine the teachings of Yamashita and Rangarajan as both inventions 
relate to document image analysis. Adding the teaching of Rangarajan provides the 
benefit of manually defining image regions. 

Yamashita also fails to teach receiving a user input indicative of region type and 
region modality specifications. However, Revankar teaches image segmentation 
according to classes of regions that may be rendered according to the same imaging 
techniques. Image regions may be rendered according to a three-class system (such as 
traditional text, graphic and picture systems), or according to more than three image 
classes. In addition, only two image classes may be required to render high quality draft 
or final output images. The image characteristics that may be rendered differently from 
class to class may include halftoning, colorization and other image attributes (see 
Abstract). It would have been obvious to one of ordinary skill in the art at the time of 
invention to combine the teachings of Yamashita and Revankar as both inventions 
relate to image segmentation. Adding the teaching of Revankar provides the benefit of 
recognizing region types by class and by modality (color, bit depth, etc.). 
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In regard to dependent Claim 9 (and similarly dependent Claim 55), 

Yamashita teaches receiving a user input indicative of vertices of a polygonal region on 
the image. However, Ranqaraian teaches input of vertices to define an image region 
(Col. 9, lines 1 5-37). It would have been obvious to one of ordinary skill in the art at the 
time of invention to combine the teachings of Yamashita and Ranqaraian as both 
inventions relate to document image analysis. Adding the teaching of Ranqaraian 
provides the benefit of manually defining image regions. 

Yamashita also fails to teach receiving a user input indicative of region type and 
region modality specifications of the polygonal region. However, Revankar teaches 
image segmentation according to classes of regions that may be rendered according to 
the same imaging techniques. Image regions may be rendered according to a three- 
class system (such as traditional text, graphic and picture systems), or according to 
more than three image classes. In addition, only two image classes may be required to 
render high quality draft or final output images. The image characteristics that may be 
rendered differently from class to class may include halftoning, colorization and other 
image attributes (see Abstract). It would have been obvious to one of ordinary skill in 
the art at the time of invention to combine the teachings of Yamashita and Revankar as 
both inventions relate to image segmentation. Adding the teaching of Revankar 
provides the benefit of recognizing region types by class and by modality (color, bit 
depth, etc.). 

In regard to dependent Claim 10 (and similarly dependent Claim 56), 

Yamashita fails to teach receiving a user input indicative of a first vertex and a location 
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of a second vertex opposite the first vertex of a rectangular region on the image. 
However, Ranqaraian teaches input of vertices to define an image region (Col. 9, lines 
15-37). It would have been obvious to one of ordinary skill in the art at the time of 
invention to combine the teachings of Yamashita and Ranqaraian as both inventions 
relate to document image analysis. Adding the teaching of Ranqaraian provides the 
benefit of manually defining image regions. 

Yamashita also fails to teach receiving a user input indicative of region type and 
region modality specifications of the rectangular region. However, Revankar teaches 
image segmentation according to classes of regions that may be rendered according to 
the same imaging techniques. Image regions may be rendered according to a three- 
class system (such as traditional text, graphic and picture systems), or according to 
more than three image classes. In addition, only two image classes may be required to 
render high quality draft or final output images. The image characteristics that may be 
rendered differently from class to class may include half toning, colorization and other 
image attributes (see Abstract). It would have been obvious to one of ordinary skill in 
the art at the time of invention to combine the teachings of Yamashita and Revankar as 
both inventions relate to image segmentation. Adding the teaching of Revankar 
provides the benefit of recognizing region types by class and by modality (color, bit 
depth, etc.). 
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12. Claims 16, 20, 61, and 64 remain rejected under 35 U.S.C. 103(a) as being 
unpatentable over Yamashita in view of Rangarajan, and in further view of Mahoney et 
al. (hereinafter Mahoney, U.S. Patent No. 5,999,664 filed 11/14/1997, issued 
12/07/1999). 

In regard to dependent Claim 16, Yamashita teaches receiving definition of at 
least one region comprises verifying the user-specified region location and boundaries 
conform to at least one region management model. However, Mahoney teaches 
searching and identifying documents based on their makeup (structure, content, etc.). 
Their system performs structural analysis at two levels. At the lower level, specific 
layout formats of a document can be identified (e.g., the recipient field of a letter or the 
header field of a memo). Such identification is performed herein using features. At the 
higher level, the entire configuration of an input document is captured using genre 
models. For example, a "business letter" is a genre model of a document that can be 
defined in most instances by a letter-date feature, a letter-recipient feature, a letter-cc 
feature, and a letter-signature feature (as shown in Fig. 3). Although some models may 
have some features in common, such models may still be distinguishable from each 
other by either the presence or absence of other features. For example, a memo and a 
letter may have similar letter-signature features while each may have different 
document header features (e.g., four-memo mark and letter-recipient) (Col. 20, lines 45- 
62).. It would have been obvious to one of ordinary skill in the art at the time of invention 
to combine the teachings of Yamashita and Mahoney as both inventions relate to 
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comparing document images to models or templates of documents. Adding the teaching 
of Mahonev provides the benefit of identifying documents (or regions thereof) with 
layout models. 

In regard to dependent Claim 20 (and similarly dependent Claim 64), 

Yamashita fails to teach determining whether the user-specified region boundaries fall 
within the visible area. However, Mahonev teaches searching and identifying 
documents based on their makeup (structure, content, etc.). Their system performs 
structural analysis at two levels. At the lower level, specific layout formats of a 
document can be identified (e.g., the recipient field of a letter or the header field of a 
memo). Such identification is performed herein using features. At the higher level, the 
entire configuration of an input document is captured using genre models. For example, 
a "business letter" is a genre model of a document that can be defined in most 
instances by a letter-date feature, a letter-recipient feature, a letter-cc feature, and a 
letter-signature feature (as shown in Fig. 3). Although some models may have some 
features in common, such models may still be distinguishable from each other by either 
the presence or absence of other features. For example, a memo and a letter may have 
similar letter-signature features while each may have different document header 
features (e.g., four-memo mark and letter-recipient) (Col. 20, lines 45-62). It would have 
been obvious to one of ordinary skill in the art at the time of invention to combine the 
teachings of Yamashita and Mahonev as both inventions relate to comparing document 
images to models or templates of documents. Adding the teaching of Mahonev provides 
the benefit of identifying documents (or regions thereof) with layout models. 
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In regard to dependent Claim 61, Claim 61 contains subject matter similar to 
that found in Claims 15 and 16, and is rejected along similar lines of reasoning. 

13. Claims 18-19, 62-63 remain rejected under 35 U.S.C. 103(a) as being 
unpatentable over Yamashita in view of Rangarajan, and in further view of Mahoney, 
and in further view of Taylor et al. (hereinafter Taylor, U.S. Patent No. 5,848,184 filed 
06/30/1995, issued 12/08/1998). 

In regard to dependent Claim 18-19 (and similarly dependent Claims 62-63), 
Yamashita fails to teach determining whether the user-specified region boundaries 
overlap with another region. However, Taylor teaches detection of overlapping 
boundaries as well as bounding boxes, which cross one another (Col. 7, lines 36-63). 

14. Claims 21, and 65 remain rejected under 35 U.S.C. 103(a) as being unpatentable 
over Yamashita in view of Rangarajan, and in further view of Ahlstrom et al. (hereinafter 
Ahlstrom, U.S. Patent No. 6,594,030 filed 08/27/1999, issued 07/15/2003). 

In regard to dependent Claim 21 (and similarly dependent Claims 42, and 
65), Yamashita fails to teach determining whether the user-specified region comply with 
a predetermined multiple z-order specification. However, Ahlstrom teaches z-order as it 
relates to how pages are overlapped upon one another (Col. 6, lines 23-56). It would 
have been obvious to one of ordinary skill in the art at the time of invention to combine 
the teachings of Yamashita and Ahlstrom as both inventions relate to analysis of page 
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objects. Adding the teaching of Ahlstrom provides the benefit of checking z-ordering of 
pages. 

15. Claims 22-24, 33-35, and 43-47 remain rejected under 35 U.S.C. 103(a) as being 
unpatentable over Yamashita in view of Mahoney. 

In regard to independent Claim 22, Claim 22 reflects the method of Claim 1 
and is rejected along the same rationale. 

In addition, Yamashita fails to teach searching for an image layout definition 
template that best matches the generated image layout definition; and conforming the 
generated image layout definition to the best-matched image layout definition template. 
However, Mahonev teaches searching and identifying documents based on their 
makeup (structure, content, etc.). Their system performs structural analysis at two 
levels. At the lower level, specific layout formats of a document can be identified (e.g., 
the recipient field of a letter or the header field of a memo). Such identification is 
performed herein using features. At the higher level, the entire configuration of an input 
document is captured using genre models. For example, a "business letter" is a genre 
model of a document that can be defined in most instances by a letter-date feature, a 
letter-recipient feature, a letter-cc feature, and a letter-signature feature (as shown in 
Fig. 3). Although some models may have some features in common, such models may 
still be distinguishable from each other by either the presence or absence of other 
features. For example, a memo and a letter may have similar letter-signature features 
while each may have different document header features (e.g., four-memo mark and 
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letter-recipient) (Col. 20, lines 45-62). It would have been obvious to one of ordinary skill 
in the art at the time of invention to combine the teachings of Yamashita and Mahonev 
as both inventions relate to comparing document images to models or templates of 
documents. Adding the teaching of Mahonev provides the benefit of identifying 
documents (or regions thereof) with layout models. 

In regard to dependent Claim 23, Yamashita teaches displaying the image on a 
display (Col. 3, lines 53-59; Fig. 6). 

In regard to dependent Claim 24, Claim 24 reflects the method as claimed in 
Claim 1 (and similarly Claim 48), and is rejected along the same rationale. 

In regard to dependent Claims 33-35, Yamashita fails to teach such limitations 
for manipulating scanned document images for purposes of displaying or transmitting in 
order to provide an image that is the appropriate size, dimension, color depth for the 
given action. However, such functions were known and obvious to one of ordinary skill 
in the art at the time of invention particularly with respect to graphical user interfaces 
where one would have desired to view entire images on a screen independent of the 
size of the actual image for purposes such as identifying regions of interest. 

In regard to dependent Claims 43-45, and 47, Yamashita fails to explicitly 
teach adjusting the location, type, modality, or visible area specification of the at least 
one region of the image layout definition. However Mahonev teaches a document 
search system provides a user with a programming interface for dynamically specifying 
features of documents recorded in a corpus of documents (Abstract). Mahonev provides 
a user interface which allows for the definition or adjustment of a given documents' 
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parameters in order to search a corpus of documents looking for similarities. Thus, it 
would have been obvious to one of ordinary skill in the art at the time of invention to use 
the user interface of Mahonev to make adjustments in the model of a current document 
to make identification of all or a part of similar documents more likely. It also would have 
been obvious to one of ordinary skill in the art at the time of invention to combine the 
teachings of Yamashita and Mahonev as both inventions relate to comparing document 
images to models or templates of documents. Adding the teaching of Mahonev provides 
the benefit of identifying documents (or regions thereof) with layout models. 

In regard to dependent Claim 46, Claim 46 contains subject matter similar to 
that found in Claim 1 (and similarly Claim 48), and is rejected along similar lines of 
reasoning. 

16. Claims 25 and 32 remain rejected under 35 U.S.C. 103(a) as being unpatentable 
over Yamashita in view of Mahoney, and in further view of Revankar. 

In regard to dependent Claim 25, Yamashita fails to teach receiving a definition 
of at least one region in an image further comprises receiving a modality specification. 
However, Revankar teaches image segmentation according to classes of regions that 
may be rendered according to the same imaging techniques. Image regions may be 
rendered according to a three-class system (such as traditional text, graphic and picture 
systems), or according to more than three image classes. In addition, only two image 
classes may be required to render high quality draft or final output images. The image 
characteristics that may be rendered differently from class to class may include half 
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toning, colorization and other image attributes (see Abstract). It would have been 
obvious to one of ordinary skill in the art at the time of invention to combine the 
teachings of Yamashita and Revankar as both inventions relate to image segmentation. 
Adding the teaching of Revankar provides the benefit of recognizing region types by 
class and by modality (color, bit depth, etc.). 

In regard to dependent Claim 32, Yamashita fails to teach receiving a user 
input indicative of a first vertex and a location of a second vertex opposite the first 
vertex of the visible area on the image. However, Revankar teaches image 
segmentation according to classes of regions that may be rendered according to the 
same imaging techniques. Image regions may be rendered according to a three-class 
system (such as traditional text, graphic and picture systems), or according to more 
than three image classes. In addition, only two image classes may be required to render 
high quality draft or final output images. The image characteristics that may be rendered 
differently from class to class may include half toning, colorization and other image 
attributes (see Abstract). It would have been obvious to one of ordinary skill in the art at 
the time of invention to combine the teachings of Yamashita and Revankar as both 
inventions relate to image segmentation. Adding the teaching of Revankar provides the 
benefit of recognizing region types by class and by modality (color, bit depth, etc.). 
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17. Claim 26 remains rejected under 35 U.S.C. 103(a) as being unpatentable over 
Yamashita in view of Mahoney, and in further view of Sakai. 

In regard to dependent Claim 26, Claim 26 reflects the method of Claims 4 and 
5 and is rejected along the same rationale. 

18. Claim 27 remains rejected under 35 U.S.C. 103(a) as being unpatentable over 
Yamashita in view of Mahoney, and in further view of Ohta. 

In regard to dependent Claim 27, Yamashita fails to teach receiving a user 
input indicative of a point on the image; and defining a region encompassing the point 
using segmentation and classification analyses of the image. However, Ohta teaches 
scanning a document, rendering it to a touch display, and allowing the user to manually 
select a region or regions to further process; the drawing of a box is done automatically 
(Col. 7, lines 46-67; Col. 8, lines 1-2). It would have been obvious to one of ordinary skill 
in the art at the time of invention to combine the teachings of Yamashita and Ohta as 
both inventions relate to designating regions of documents for further analysis. Adding 
the teaching of Ohta provides the user with a means to easily choose which portions of 
a document to further analyze. 
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19. Claims 28, 36-37, and 41 remain rejected under 35 U.S.C. 103(a) as being 
unpatentable over Yamashita in view of Mahoney, and in further view of Rangarajan. 

In regard to dependent Claim 28, Yamashita fails to teach receiving a user 
input indicative of boundaries of the region on the image; and receiving a user input 
indicative of region type and region modality specifications. However, Ranqaraian 
teaches a conventional set of drawing-like tools with which the user can graphically 
create 31 1 the user defined zones. This is done by choosing an appropriate drawing 
tool, such as a rectangle or polygon creation tool, and applying it to the de-skewed 
image to select the individual areas or zones containing the desired text information. 
Fig. 7a illustrates one example of a suitable user interface 705, showing a de-skewed 
document 700. Fig. 7b illustrates the same document now including a number of user- 
defined zones 701. A palette of drawing tools 703 is also shown, with various graphical 
tools for selecting the user-defined zones 701 . Once the user defines a number of 
zones, the coordinates of the boundary of each of user defined zone is stored, 
preferably using the coordinates of an upper left hand corner, and a lower right hand 
corner where the user defined zone is a rectangle. For general polygonal user defined 
zones, the coordinates of each vertex may also be stored (Col. 9, lines 15-37; Figs. 7A- 
B). It would have been obvious to one of ordinary skill in the art at the time of invention 
to combine the teachings of Yamashita and Ranqaraian as both inventions relate to 
document image analysis. Adding the teaching of Ranqaraian provides the benefit of 
manually defining image regions. 
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In regard to dependent Claim 36, Yamashita fails to teach receiving definition 
of at least one region comprises receiving a user specification of a location and 
boundaries of a region in the image. However, Ranqaraian teaches input of vertices to 
define an image region (Col. 9, lines 15-37). It would have been obvious to one of 
ordinary skill in the art at the time of invention to combine the teachings of Yamashita 
and Ranqaraian as both inventions relate to document image analysis. Adding the 
teaching of Ranqaraian provides the benefit of manually defining image regions. 

In regard to dependent Claim 37, Yamashita teaches receiving definition of at 
least one region comprises verifying the user-specified region location and boundaries 
conform to at least one region management model. However, Mahonev teaches 
searching and identifying documents based on their makeup (structure, content, etc.). 
Their system performs structural analysis at two levels. At the lower level, specific 
layout formats of a document can be identified (e.g., the recipient field of a letter or the 
header field of a memo). Such identification is performed herein using features. At the 
higher level, the entire configuration of an input document is captured using genre 
models. For example, a "business letter" is a genre model of a document that can be 
defined in most instances by a letter-date feature, a letter-recipient feature, a letter-cc 
feature, and a letter-signature feature (as shown in Fig. 3). Although some models may 
have some features in common, such models may still be distinguishable from each 
other by either the presence or absence of other features. For example, a memo and a 
letter may have similar letter-signature features while each may have different 
document header features (e.g., four-memo mark and letter-recipient) (Col. 20, lines 45- 
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62). It would have been obvious to one of ordinary skill in the art at the time of invention 
to combine the teachings of Yamashita and Mahonev as both inventions relate to 
comparing document images to models or templates of documents. Adding the teaching 
of Mahonev provides the benefit of identifying documents (or regions thereof) with 
layout models. 

In regard to dependent Claim 41, Yamashita fails to teach determining whether 
the user-specified region boundaries fall within the visible area. However, Mahonev 
teaches searching and identifying documents based on their makeup (structure, 
content, etc.). Their system performs structural analysis at two levels. At the lower level, 
specific layout formats of a document can be identified (e.g., the recipient field of a letter 
or the header field of a memo). Such identification is performed herein using features. 
At the higher level, the entire configuration of an input document is captured using 
genre models. For example, a "business letter" is a genre model of a document that can 
be defined in most instances by a letter-date feature, a letter-recipient feature, a letter- 
cc feature, and a letter-signature feature (as shown in Fig. 3). Although some models 
may have some features in common, such models may still be distinguishable from 
each other by either the presence or absence of other features. For example, a memo 
and a letter may have similar letter-signature features while each may have different 
document header features (e.g., four-memo mark and letter-recipient) (Col. 20, lines 45- 
62). It would have been obvious to one of ordinary skill in the art at the time of invention 
to combine the teachings of Yamashita and Mahonev as both inventions relate to 
comparing document images to models or templates of documents. Adding the teaching 
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of Mahonev provides the benefit of identifying documents (or regions thereof) with 
layout models. 

20. Claims 29-31, and 38 remain rejected under 35 U.S.C. 103(a) as being 
unpatentable over Yamashita in view of Rangarajan, and in further view of Revankar. 

In regard to dependent Claim 29, Yamashita fails to teach receiving a user 
input indicative of vertices of the region on the image. However, Rangarajan teaches 
input of vertices to define an image region (Col. 9, lines 15-37). It would have been 
obvious to one of ordinary skill in the art at the time of invention to combine the 
teachings of Yamashita and Ranaaraian as both inventions relate to document image 
analysis. Adding the teaching of Rangarajan provides the benefit of manually defining 
image regions. 

Yamashita also fails to teach receiving a user input indicative of region type and 
region modality specifications. However, Revankar teaches image segmentation 
according to classes of regions that may be rendered according to the same imaging 
techniques. Image regions may be rendered according to a three-class system (such as 
traditional text, graphic and picture systems), or according to more than three image 
classes. In addition, only two image classes may be required to render high quality draft 
or final output images. The image characteristics that may be rendered differently from 
class to class may include half toning, colorization and other image attributes (see 
Abstract). It would have been obvious to one of ordinary skill in the art at the time of 
invention to combine the teachings of Yamashita and Revankar as both inventions 
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relate to image segmentation. Adding the teaching of Revankar provides the benefit of 
recognizing region types by class and by modality (color, bit depth, etc.). 

In regard to dependent Claim 30, Yamashita teaches receiving a user input 
indicative of vertices of a polygonal region on the image. However, Ranaaraian teaches 
input of vertices to define an image region (Col. 9, lines 15-37). It would have been 
obvious to one of ordinary skill in the art at the time of invention to combine the 
teachings of Yamashita and Ranaaraian as both inventions relate to document image 
analysis. Adding the teaching of Ranaaraian provides the benefit of manually defining 
image regions. 

Yamashita also fails to teach receiving a user input indicative of region type and 
region modality specifications of the polygonal region. However, Revankar teaches 
image segmentation according to classes of regions that may be rendered according to 
the same imaging techniques. Image regions may be rendered according to a three- 
class system (such as traditional text, graphic and picture systems), or according to 
more than three image classes. In addition, only two image classes may be required to 
render high quality draft or final output images. The image characteristics that may be 
rendered differently from class to class may include halftoning, colorization and other 
image attributes (see Abstract). It would have been obvious to one of ordinary skill in 
the art at the time of invention to combine the teachings of Yamashita and Revankar as 
both inventions relate to image segmentation. Adding the teaching of Revankar 
provides the benefit of recognizing region types by class and by modality (color, bit 
depth, etc.). 
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In regard to dependent Claim 31. Yamashita fails to teach receiving a user 
input indicative of a first vertex and a location of a second vertex opposite the first 
vertex of a rectangular region on the image. However, Ranqaraian teaches input of 
vertices to define an image region (Col. 9, lines 15-37). It would have been obvious to 
one of ordinary skill in the art at the time of invention to combine the teachings of 
Yamashita and Ranqaraian as both inventions relate to document image analysis. 
Adding the teaching of Ranqaraian provides the benefit of manually defining image 
regions. 

Yamashita also fails to teach receiving a user input indicative of region type and 
region modality specifications of the rectangular region. However, Revankar teaches 
image segmentation according to classes of regions that may be rendered according to 
the same imaging techniques. Image regions may be rendered according to a three- 
class system (such as traditional text, graphic and picture systems), or according to 
more than three image classes. In addition, only two image classes may be required to 
render high quality draft or final output images. The image characteristics that may be 
rendered differently from class to class may include halftoning, colorization and other 
image attributes (see Abstract). It would have been obvious to one of ordinary skill in 
the art at the time of invention to combine the teachings of Yamashita and Revankar as 
both inventions relate to image segmentation. Adding the teaching of Revankar 
provides the benefit of recognizing region types by class and by modality (color, bit 
depth, etc.). 
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In regard to dependent Claim 38, Yamashita fails to teach receiving user 
specification of region type and region modality. However, Revankar teaches image 
segmentation according to classes of regions that may be rendered according to the 
same imaging techniques. Image regions may be rendered according to a three-class 
system (such as traditional text, graphic and picture systems), or according to more 
than three image classes. In addition, only two image classes may be required to render 
high quality draft or final output images. The image characteristics that may be rendered 
differently from class to class may include half toning, colorization and other image 
attributes (see Abstract). It would have been obvious to one of ordinary skill in the art at 
the time of invention to combine the teachings of Yamashita and Revankar as both 
inventions relate to image segmentation. Adding the teaching of Revankar provides the 
benefit of recognizing region types by class and by modality (color, bit depth, etc.). 

21 . Claims 39-40 remain rejected under 35 U.S.C. 1 03(a) as being unpatentable 
over Yamashita in view of Mahoney, and in further view of Rangarajan, and in further 
view of Taylor. 

In regard to dependent Claim 39-40, Yamashita fails to teach determining 
whether the user-specified region boundaries overlap with another region. However, 
Taylor teaches detection of overlapping boundaries as well as bounding boxes, which 
cross one another (Col. 7, lines 36-63). 
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22. Claim 42 remains rejected under 35 U.S.C. 103(a) as being unpatentable over 
Yamashita in view of Mahoney, and in further view of Rangarajan, and in further view of 
Ahlstrom. 

In regard to dependent Claim 42, Yamashita fails to teach determining whether 
the user-specified region comply with a predetermined multiple z-order specification. 
However, Ahlstrom teaches z-order as it relates to how pages are overlapped upon one 
another (Col. 6, lines 23-56). It would have been obvious to one of ordinary skill in the 
art at the time of invention to combine the teachings of Yamashita and Ahlstrom as both 
inventions relate to analysis of page objects. Adding the teaching of Ahlstrom provides 
the benefit of checking z-ordering of pages. 
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Response to Arguments 

23. Applicants argue that the prior art of Yamashita does not disclose or even 
suggest the limitation of Claim 1 (and as amended, similarly Claim 48) of receiving a 
definition of a visible area in the image, the visible area definition having a specification 
of margins around the image . The Examiner respectfully disagrees. Yamashita does 
indeed disclose this limitation in that the visible area(s) in Yamashita are clearly defined 
by margins surrounding each of the segmented areas (Figs. 3, 16, 17, 20; depict 
margins surrounding visible areas of the page, the margins define the segments). These 
segmented areas are then used to produce a model of the document layout. 

24. Applicant further argues that the prior art of Mahonev in combination with 
Yamashita fails to disclose the limitation of Claim 22 of conforming the generated image 
layout definition to the best-matched image layout definition template . The Examiner 
respectfully disagrees. Mahonev allows a user to define a model layout and to then 
query a corpus of documents looking for those with similar layouts. Mahonev was 
introduced to cover the limitations of searching for and finding the best match between 
the layout model and existing layouts identified in a document corpus. 

25. Further, Applicants respectfully submit that there is no motivation or suggestion 
to combine the purported teachings of Mahonev with Yamashita as proposed by the 
Examiner. For example, in the Office Action, as a basis for combining the purported 
teachings of Mahonev with Yamashita . the Examiner states that it provides the benefit 
of identifying documents (or regions thereof) with models (Office Action, page 17). As 
discussed above in connection with the rejection under 35 U.S.C. § 102 of independent 
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Claims 1 and 48, the Examiner appears to indicate that Yamashita already discloses 
identifying and/or defining regions in an image (Office Action, pages 2 and 3). Thus, 
there appears to be no motivation or suggestion to combine the purported teachings of 
Mahonev with Yamashita at least because, at least according to the Examiner, 
Yamashita appears to already disclose what the Examiner is relying on in Mahoney. 
Accordingly, for at least this reason also, Applicants respectfully submit that Claim 22 is 
patentable over the Yamashita and Mahonev references. The Examiner respectfully 
responds by saying that what was intended to be said was that Yamashita and 
Mahonev relate to generating layouts of documents through the use of models. 
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Conclusion 

26. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .1 36(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

27. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to James H. Blackwell whose telephone number is 571- 

272- 4089. The examiner can normally be reached on Mon-Fri. 

28. If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Heather R. Herndon can be reached on 571-272-4136. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 

273- 8300. 
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29. Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

James H. Blackwell 
09/11/2006 




